Search for: All records

The complex and interdisciplinary nature of scientific concepts presents formidable challenges for students in developing their knowledge-in-use skills. The utilization of computerized analysis for evaluating students’ contextualized constructed responses offers a potential avenue for educators to develop personalized and scalable interventions, thus supporting the current teaching and learning of science. While prior research in artificial intelligence has demonstrated the effectiveness of algorithms, including Bidirectional Encoder Representations from Transformers (BERT), in tasks like automated classifications of constructed responses, these efforts have predominantly leaned towards text-level features, often overlooking the exploration of conceptual ideas embedded in students’ responses from a cognitive perspective. Despite BERT’s performance in downstream tasks, challenges may arise in domain-specific tasks, particularly in establishing knowledge connections between specialized and open domains. These challenges become pronounced in small-scale and imbalanced educational datasets, where the available information for fine-tuning is frequently inadequate to capture task-specific nuances and contextual details. The primary objective of the present study is to investigate the effectiveness of a pretrained language model, when integrated with an ontological framework aligned with a contextualized science assessment, in classifying students’ expertise levels in scientific explanation. Our findings indicate that while pretrained language models, such as BERT, contribute to enhanced performance in language-related tasks within educational contexts, the incorporation of identifying domain-specific terms and extracting and substituting with their associated sibling terms in sentences through ontology-based systems can significantly improve classification model performance. Further, we qualitatively examined student responses and found that, as expected, the ontology framework identified and substituted key domain-specific terms in student responses that led to more accurate predictive scores. The study explores the practical implementation of ontology in assessment evaluation to facilitate formative assessment and formulate instructional strategies. 

Abstract As future decision-makers, students must develop interdisciplinary, systems thinking skills to make effective management decisions; however, systems thinking remains challenging for many students. Here, we use the Food-Energy-Water (FEW) Nexus as a framework to examine how drawings can help students cultivate systems thinking skills. Drawings can be tools to make implicit mental models of systems connections explicit for instructors to better comprehend student learning. Our goal was to understand how drawing can help students make connections across systems compared to using only verbal explanations. In 2021, we interviewed undergraduates, asking them to draw and verbally explain the FEW Nexus. Analysis revealed that student drawings showed an increase in the number of connections that half of students could describe when compared to verbal-only explanations. Instructors may benefit from this study by recognizing areas where students might struggle to understand FEW Nexus connections, where additional course emphasis is needed, and how drawings can help assess student learning. 

Interdisciplinary environmental and sustainability (IES) programs are different from other fields because they focus on a complex integration of humanities, social, and natural sciences concepts centered on the interactions of coupled human and natural systems. The interdisciplinary nature of IES programs does not lend itself to traditional discipline-specific concept inventory frameworks for critically evaluating preconceptions and learning. We discuss the results of the first phase of a research project to develop a next generation concept inventory for evaluating interdisciplinary concepts important for introductory IES courses. Using the Food-Energy-Water (FEW) Nexus (the intersections/interdependencies of food, energy, and water sectors) as our focus, we conducted a content analysis of eight representative college-level introductory environmental course syllabi and course materials (e.g., textbooks, journal articles, print media) to identify common interdisciplinary FEW Nexus concepts taught in introductory IES courses. Results demonstrate that all IES introductory course materials reference the FEW Nexus. Food, energy, and/or water resources as individual elements of the FEW Nexus are frequently described, but connections between these resource systems are included less often. Biology, energy systems, waste and pollution in the natural environment, agriculture, earth sciences and geology, climate change, behavioral social sciences, and economics concepts are most associated with FEW concepts, hinting at commonalities across IES topics that anchor systems thinking. Despite differences in IES programs, there appears to be some alignment between core concepts being taught at the FEW Nexus in introductory courses.